Distribution Fields with Adaptive Kernels for Large Displacement Image Alignment

While region-based image alignment algorithms that use gradient descent can achieve sub-pixel accuracy when they converge, their convergence depends on the smoothness of the image intensity values. Image smoothness is often enforced through the use of multi-scale approaches in which images are smoothed and downsampled. Yet, these approaches typically use fixed smoothing parameters which may be appropriate for some images but not for others. Even for a particular image, the optimal smoothing parameters may depend on the magnitude of the transformation. When the transformation is large, the image should be smoothed more than when the transformation is small. Further, with gradient-based approaches, the optimal smoothing parameters may change with each iteration as the algorithm proceeds towards convergence. We address convergence issues related to the choice of smoothing parameters by deriving a Gauss-Newton gradient descent algorithm based on distribution fields (DFs) and proposing a method to dynamically select smoothing parameters at each iteration. DFs have previously been used in the context of tracking [6]. In this work we incorporate DFs into a full affine model for region-based alignment and simultaneously search over parameterized sets of geometric and photometric transforms. We use a probabilistic interpretation of DFs to select smoothing parameters at each step in the optimization and show that this results in improved convergence rates. Following the notation of Baker and Matthews [1], let T (x) be an image containing a fixed region for which we want to find the corresponding region in I(x), where x = (x,y)T is a column vector. We refer to T (x) as the template image and I(x) as the input image. Let W (x;p) be the parameterized set of warps, where p are the parameters. The goal of regionbased alignment is to find the p̂ that minimizes some distance measure between T (x) and I(W(x; p̂)). One of the early image alignment algorithms was the gradient-based Lucas-Kanade (LK) algorithm [3]. In the LK method, the feature space consists of intensity values and the similarity measure is the L2 distance. Our method is inspired by the LK algorithm, but we replace the image and template with their DF representations. The basic idea of a DF is to represent a region in an image as a normalized histogram, i.e., a probability distribution, over feature values at each pixel. In this work we use grayscale intensity values as the feature values although other features could be used instead (e.g. edge intensities, RGB values for color images, etc.). The simplest DF consists of probability distributions over binned intensity values where each probability distribution is degenerate and is given by